The ability to successfully operate in darkness is a cornerstone of our military superiority. This capability is supported by a number of sensor technologies that outfit the modern dismounted soldier. Ranging from intensified near infrared (NIR), through shortwave infrared (SWIR), midwave infrared (MWIR) and onto the longwave thermal infrared (LWIR), these technologies allow for unprecedented sensory enhancement and extended operational conditions. One negative aspect of the proliferation of night-vision sensors is that even though many of these devices provide complementary capabilities, they do not build upon each other, and rather function separately. This result in redundant equipment, inability to simultaneously exploit more than one sensor modality and overly cumbersome operation, leading to decreased efficiency and higher vulnerability.
Simultaneous, complementary use of disparate sensor modalities would provide the dismounted soldier with increased situational awareness, and would allow for optimal use of available resources. The lower complexity of a unified sensor/display package would free the user to concentrate on the mission, not the tools, while at the same time having better quality sensory input upon which to operate. In addition to providing a combined output from multiple sensors, a unified system would obviate the awkward transition between devices that is currently necessary. With the proposed system, the user would be able to transition from one sensor output to another, or to a combination of multiple sensors with no need to switch or rearrange equipment. This would address a critical transition time during which the user is both blind and distracted, and therefore more vulnerable. A unified system would allow the user to have continuous, seamless visual input from multiple complementary sensors, greatly enhancing his capabilities.
There is an operational gap in being able to seamlessly transition/coordinate between a rifle mounted thermal weapon sight (TWS) and helmet mounted electro-optical (EO) cameras (e.g., intensified imaging). A soldier looking through a TWS temporarily interrupts his ability to perceive his surroundings beyond the narrow field-of-view of the TWS which can be fatal in a battlefield environment. In other instances, a soldier does not precisely know where his rifle is aimed in his visible field-of-view without peering into the TWS. In Low-light/No-Light conditions infantry have little to virtually no natural visual perception relying heavily upon intensified EO and thermal cameras provided to them. While on-the-move in a dynamic battlefield environment the soldier should be able to persistently perceive their surroundings through intensified EO while at the same time being able to direct their rifle aim point and accurately “shoot from the hip” with minimal distraction from their situational awareness. Furthermore this can be done passively without the use of active laser pointers or designators.
Current operations have determined that intensified night vision goggles alone do not provide the soldier with the capability to operate and survive in many environmental conditions and mission scenarios. As the enemy gets smarter in defeating night vision goggles and continues to gain this capability themselves US troops are finding their technological advantage deteriorating. Because of this, various DOD programs are emerging that emphasize the complementary use of EO and thermal infrared modalities and respective fusion.
A portion of the invention uses Inertial Measurement Units (IMU) consisting of a combination of accelerometers, gyroscopes and magnetometers. The accelerometers are used to determine the vertical direction, by measuring the acceleration of gravity. The magnetometers use the Earth's magnetic field to determine magnetic North, and thus left-to-right orientation. Finally, the gyroscopes are used to stabilize the instantaneous measurements from the other devices, and neutralize jitter from sensor noise. One problem with such a system is that the magnetometers are interfered upon by existing environmental magnetic fields, as well as the presence of large ferromagnetic objects. This is especially a problem when we consider that weapons themselves are made of steel, not to mention armored vehicles and other equipment normally found in the battlefield. The immediate implication is that any system based on magnetic readings will suffer interference from environmental noise that may render the entire system inaccurate. The novelty of the current invention is that it overcomes the inaccuracy limitations of IMUs by using both IMU information, and image feature registration through image processing to produce a much more reliable system.